Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Abstract Value Analysis #1759

Open
wants to merge 61 commits into
base: main
Choose a base branch
from
Open

Abstract Value Analysis #1759

wants to merge 61 commits into from

Conversation

CodingDepot
Copy link
Collaborator

@CodingDepot CodingDepot commented Oct 2, 2024 •
edited
Loading

This PR tracks the implementation of an improved abstract value analysis.
The evaluator aims to determine possible values of a given variable at the specified node by considering all changes made to it starting from its initialization.

It aims to support multiple different types of values including simple number values as well as the length of collections. For this, the supported values need to define themselves which operations performed on them affect their possible value range.

The end result is an interval spanning the range of possible values for the variable at the specified node.
The initial PR focuses on Integer evaluation, but it can be easily extended to support other cases.

Open Work:

  • Integrate the existing EOGWorklist into the analysis
  • Write Unit Tests for different code patterns
  • Write MORE tests to increase coverage

Moved out of scope for the initial PR

  • Pre-Evaluate conditions for better performance and result
  • Add additional supported Values

CodingDepot added 19 commits July 24, 2024 12:22
@CodingDepot
create initial Size evaluator for modifiable lists
3e76a77
@CodingDepot
move all collection dependant logic into collection class
7d9db53
@CodingDepot
add support for static arrays
3abfa1c
@CodingDepot
implement narrowing and widening operations
cb55e93
@CodingDepot
add narrowing and widening to loop analysis
f43919d
@CodingDepot
handle branches in analysis
901f9c2
@CodingDepot
improve loops by preprocessing, better narrowing and valid starting r…
4f0e95a 
…anges
@CodingDepot
improve subtraction by handling 0 as lowest possible value
34412ba
@CodingDepot
mark branching nodes as nodes with an effect
372572c
@CodingDepot
correctly identify nodes of deeper branch layers in a loop
0e3bc5b
@CodingDepot
add documentation
a002201
@CodingDepot
temporarily repurpose cpg-neo4j for debugging
3648081
@CodingDepot
rework lattice intervals to support negative values
899f919
@CodingDepot
rename target package to circumvent gitignore
ecc8035
@CodingDepot
implement comparable for the LatticeInterval
c6122c6
@CodingDepot
implement the IntervalState
fb884f5
@CodingDepot
update the evaluator name in tests
c6d3abb
@CodingDepot
implement equals check for intervals
d231c6d
@CodingDepot
add method stub for condition evaluation
54210d9
@CodingDepot CodingDepot added the analysis Concerns CPG analysis module label Oct 2, 2024
@CodingDepot CodingDepot requested a review from KuechA October 2, 2024 09:09
@CodingDepot CodingDepot self-assigned this Oct 2, 2024
@CLAassistant
Copy link

CLAassistant commented Oct 2, 2024 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

KuechA and others added 7 commits October 3, 2024 10:31
@KuechA
Merge branch 'main' into rh/abstract-value-analysis
ed21aa8
@KuechA
Spotless
344ea58
@CodingDepot
Squashed commit of the following:
52cb857 
commit 5c20b05
Author: Robert Haimerl <robert.haimerl@aisec.fraunhofer.de>
Date:   Wed Oct 16 13:25:06 2024 +0200

    fix breaking merge changes

commit cc8eb48
Merge: 3e3c094 344ea58
Author: Robert Haimerl <robert.haimerl@aisec.fraunhofer.de>
Date:   Wed Oct 16 13:08:46 2024 +0200

    Merge branch 'rh/abstract-value-analysis' into rh/abstract-value-analysis-worklist

    # Conflicts:
    #	cpg-analysis/src/main/kotlin/de/fraunhofer/aisec/cpg/analysis/abstracteval/AbstractEvaluator.kt
    #	cpg-core/src/main/kotlin/de/fraunhofer/aisec/cpg/helpers/EOGWorklist.kt

commit 3e3c094
Author: Robert Haimerl <robert.haimerl@aisec.fraunhofer.de>
Date:   Wed Oct 16 13:04:45 2024 +0200

    remove redundant pushes to the worklist itself

commit 6d731f2
Author: Robert Haimerl <robert.haimerl@aisec.fraunhofer.de>
Date:   Wed Oct 16 13:04:21 2024 +0200

    override methods to use custom functionality in IntervalStates

commit 1eb5a51
Author: Robert Haimerl <robert.haimerl@aisec.fraunhofer.de>
Date:   Wed Oct 16 13:03:57 2024 +0200

    enhance analysis for simple value operations

commit 108f374
Author: Robert Haimerl <robert.haimerl@aisec.fraunhofer.de>
Date:   Wed Oct 16 09:56:22 2024 +0200

    join intervals for multiple EOG (branch joins)

commit d4383b4
Author: Robert Haimerl <robert.haimerl@aisec.fraunhofer.de>
Date:   Wed Oct 16 09:33:41 2024 +0200

    revert the change to the worklist pop to make it FIFO again

commit 46798c5
Author: Robert Haimerl <robert.haimerl@aisec.fraunhofer.de>
Date:   Wed Oct 16 09:33:23 2024 +0200

    return a new altered state instead of directly modifying the current state

commit c634646
Author: Robert Haimerl <robert.haimerl@aisec.fraunhofer.de>
Date:   Mon Oct 14 13:12:46 2024 +0200

    remove getInitialRange from the evaluator

commit f984a2c
Author: Robert Haimerl <robert.haimerl@aisec.fraunhofer.de>
Date:   Mon Oct 14 13:11:58 2024 +0200

    remove the "getInitialRange" method for values and instead mark declarations as operations with effect

commit beb4d38
Author: Robert Haimerl <robert.haimerl@aisec.fraunhofer.de>
Date:   Mon Oct 14 12:26:22 2024 +0200

    simplify evaluator to only use one worklist without special handling for loops and branches

commit 6ec77dc
Author: Robert Haimerl <robert.haimerl@aisec.fraunhofer.de>
Date:   Mon Oct 14 12:25:04 2024 +0200

    add "until" to iteration, add state information to Worklist, fix pop order

commit 253debb
Author: Robert Haimerl <robert.haimerl@aisec.fraunhofer.de>
Date:   Mon Oct 14 12:23:17 2024 +0200

    remove all modes from the IntervalState

commit 31ca4ac
Author: Robert Haimerl <robert.haimerl@aisec.fraunhofer.de>
Date:   Mon Oct 14 12:17:45 2024 +0200

    remove boolean information about whether the operation had an impact

commit ccd839e
Author: Robert Haimerl <robert.haimerl@aisec.fraunhofer.de>
Date:   Mon Oct 14 09:17:11 2024 +0200

    add three different state modes

commit e0e1a28
Author: Robert Haimerl <robert.haimerl@aisec.fraunhofer.de>
Date:   Mon Oct 7 12:08:52 2024 +0200

    rewrite the handleBranch branch

commit 672f27f
Author: Robert Haimerl <robert.haimerl@aisec.fraunhofer.de>
Date:   Mon Oct 7 11:45:10 2024 +0200

    rewrite the handleLoop branch

commit 33f3531
Author: Robert Haimerl <robert.haimerl@aisec.fraunhofer.de>
Date:   Mon Oct 7 10:54:15 2024 +0200

    rewrite the applyEffect branch

commit ca88bb7
Author: Robert Haimerl <robert.haimerl@aisec.fraunhofer.de>
Date:   Mon Oct 7 10:20:39 2024 +0200

    rewrite evaluate function to use "iterateEOG"
@CodingDepot
fix LatticeInterval.push to join with the previous value as it is alr…
7028045 
…eady propagated from predecessors
@CodingDepot
Fix Declaration to only effect matching integers
10f2fc2
@CodingDepot
correctly propagate widening only within loops
39db0ba
@CodingDepot
added second evaluate call with more defined arguments
d74c427
@codecov Codecov
Copy link

codecov bot commented Oct 28, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 76.47059% with 116 lines in your changes missing coverage. Please review.

Project coverage is 75.23%. Comparing base (d2d7af9) to head (6557a39).
Report is 19 commits behind head on main.

Files with missing lines Patch % Lines
...aisec/cpg/analysis/abstracteval/LatticeInterval.kt 80.51% 18 Missing and 27 partials ⚠️
...sec/cpg/analysis/abstracteval/AbstractEvaluator.kt 54.16% 19 Missing and 14 partials ⚠️
...isec/cpg/analysis/abstracteval/value/ArrayValue.kt 44.44% 5 Missing and 5 partials ⚠️
...pg/analysis/abstracteval/value/MutableListValue.kt 67.74% 6 Missing and 4 partials ⚠️
...ec/cpg/analysis/abstracteval/value/IntegerValue.kt 92.50% 3 Missing and 3 partials ⚠️
...lin/de/fraunhofer/aisec/cpg/helpers/EOGWorklist.kt 80.76% 2 Missing and 3 partials ⚠️
...in/de/fraunhofer/aisec/cpg/graph/builder/Fluent.kt 86.66% 0 Missing and 4 partials ⚠️
...fer/aisec/cpg/analysis/abstracteval/value/Value.kt 40.00% 1 Missing and 2 partials ⚠️
Additional details and impacted files
Files with missing lines Coverage Δ
...fer/aisec/cpg/analysis/abstracteval/value/Value.kt 40.00% <40.00%> (ø)
...in/de/fraunhofer/aisec/cpg/graph/builder/Fluent.kt 86.03% <86.66%> (-1.18%) ⬇️
...lin/de/fraunhofer/aisec/cpg/helpers/EOGWorklist.kt 84.82% <80.76%> (+12.70%) ⬆️
...ec/cpg/analysis/abstracteval/value/IntegerValue.kt 92.50% <92.50%> (ø)
...isec/cpg/analysis/abstracteval/value/ArrayValue.kt 44.44% <44.44%> (ø)
...pg/analysis/abstracteval/value/MutableListValue.kt 67.74% <67.74%> (ø)
...sec/cpg/analysis/abstracteval/AbstractEvaluator.kt 54.16% <54.16%> (ø)
...aisec/cpg/analysis/abstracteval/LatticeInterval.kt 80.51% <80.51%> (ø)

... and 32 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@CodingDepot CodingDepot marked this pull request as ready for review October 30, 2024 11:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@oxisto oxisto oxisto left review comments

@KuechA KuechA Awaiting requested review from KuechA KuechA is a code owner

@konradweiss konradweiss Awaiting requested review from konradweiss konradweiss is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

@CodingDepot CodingDepot

Labels
analysis Concerns CPG analysis module
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@CodingDepot @CLAassistant @oxisto @KuechA